All Questions

Here’s a brief overview of what I want to achieve Extract raw htmls and save them Use Crawl4AI to produce a ‘cleaner’ and smaller HTML that has a lot of information, including what I will eventually ...

I want to use the data in my codes with Transfermark Scraper for my own special purpose. I get all the desired data in the codes except Current Club, but I can't get the Club name. I tried all the ...

I have been using Crawler4AI to try downloading a series of documents from this Website. However, since it requieres JavaScript code and I am using Python, I don't know hot to solve my error. Code, ...

I am trying to scrape data from www.example.com but the below code returns error : import asyncio from crawl4ai import AsyncWebCrawler from crawl4ai.async_configs import BrowserConfig, ...

I am seeking assistance in extracting data from a website with multiple tabs and saving it in a .csv format using Python and Selenium. The website in question is: https://www.amfiindia.com/research-...

I have two scrapy Spiders: Spider 1 crawls a list of product links (~10000) and saves them to a csv file using a feed. It doesn't visit each of those links, only the categories (with multiple pages). ...

I'm trying to crawl https://kick.com/browse/categories with playwright which has infinite scroll. I've tried evaluating the below js code and wait for an extended period for loading. I'm turning off ...

I am crawling https://kick.com/browse/categories where every time you scroll it loads new cards of a category. I have tried multiple methods using playwright but none of them worked. Would appreciate ...

Please, i need someone help me. I can't understand why I only crawl 25 movies instead of 250. My code: import pandas as pd import requests from bs4 import BeautifulSoup headers = {'User-Agent': '...

I'm building a scraper that gets text data from a list of articles. A common specimen in the text content I'm scraping at the minute is that at the bottom there is this message: "As a subscriber, ...

I am trying to download videos from a site, which requires extracting 1 "download url" that resides on each "video url". Example: "video url": https://www.example.com/...

I have a code snippet to extract the inputable and clickable node elements (i.e. interactive elements) from the DOM tree of the web pages via Playwright in python. This code almost works properly but ...

I am using icrawler on python to scrape images online. I have a list of strings download_waitlist = ["cat","dog","car","motorbike","snoop dogg"] that ...

There is a report and I want to Extract a Value from it The Value is being generated by a Component written in java script Here is particular section code being highlighted when I do Inspect element ...

I am trying to convert my web-scraper I built in JavaScript using the puppeteer library into a python-based web-scraper running on Scrapy. I want to be able to do something similar to JavaScript's ...